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Abstract — We optimize modulation formats for the additive 
white Gaussian noise channel with nonnegative input, also known 
as the intensity-modulated direct-detection channel, with and 
without confining them to a lattice structure. Our optimiza- 
tion criteria are the average electrical, average optical, and 
peak power. The nonnegative constraint on the input to the 
channel is translated into a conical constraint in signal space, 
and modulation formats are designed by sphere packing inside 
this cone. Some dense packings are found, which yield more 
power-efficient modulation formats than previously known. For 
example, at a spectral efficiency of 1.5 bit/s/Hz, the modulation 
format optimized for average electrical power has a 2.55 dB 
average electrical power gain over the best known format to 
achieve a symbol error rate of 10 -6 . The corresponding gains 
for formats optimized for average and peak optical power are 1.35 
and 1.72 dB, respectively. Using modulation formats optimized for 
peak power in average-power limited systems results in a smaller 
power penalty than when using formats optimized for average 
power in peak-power limited systems. We also evaluate the mod- 
ulation formats in terms of their mutual information to predict 
their performance in the presence of capacity-achieving error- 
correcting codes, and finally show numerically and analytically 
that the optimal modulation formats for reliable transmission in 
the wideband regime have only one nonzero point. 

Index Terms — Direct detection, fiber-optical communications, 
free-space optical communications, infrared communications, 
intensity modulation, lattice codes, mutual information, nonco- 
herent communications, sphere packing. 

I. Introduction 

COHERENT optical systems, which give access to both 
the carrier's amplitude and phase to convey information, 
allow the design of higher-order modulation formats which 
offer a good trade-off between spectral and power efficiency. 
However, in systems where phase information is absent, de- 
signing such formats becomes challenging. Examples of such 
systems include phase-noise limited systems, and noncoherent 
systems where information is encoded onto the amplitude of 
the carrier and the envelope of the received signal is detected 
at the receiver. The latter is prevalent in optical communication 
systems where the overall cost and complexity is a critical con- 
straint. This type of noncoherent systems is known as intensity- 
modulated direct-detection (IM/DD) systems and will be the 
focus of our work. In such systems, the information is encoded 
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onto the intensity of the optical carrier, and this intensity is, 
at all time instances, nonnegative. Applications using IM/DD 
are, for example, wireless optical communications Q-0 and 
short-haul fiber links used in, e.g., data centers l4lh |5l. 

In the absence of optical amplification, an IM/DD system 
can be modeled as a conventional additive white Gaussian 
noise (AWGN) channel whose input is constrained to being 
nonnegative d Ch. 5], O, El-El. Since the optical phase 
cannot be used to carry information, resorting to M-ary pulse 
amplitude modulation (M-PAM) is a natural low-complexity 
way of improving the spectral efficiency beyond that of the 
widespread on-off keying (OOK). However, this is different 
from the conventional PAM since no negative amplitudes can 
be used d Eq. (5.8)]. In OH, CD, an IM/DD link analysis 
using 4-PAM signaling was demonstrated. In O, upper and 
lower bounds on the capacity of 2-, 4-, 8-, and 16-PAM were 
derived and in IT2L the power efficiency of M-PAM was 
shown to be low. The M-ary pulse-position modulation (M- 
PPM) formats are known to be power-efficient; however, they 
suffer from poor spectral efficiency (TJ Sec. 5.3.3], O, fl3l . 

Any nonnegative electrical waveform can be communicated 
successfully over an IM/DD link. This implies that if the 
information to be transmitted is modulated on an electrical 
subcarrier using any M-level modulation format, it can be 
transmitted on an IM/DD link after adding a direct current 
(DC) bias to ensure its nonnegativity, i.e., the subcarrier 
amplitude and phase which carry the information can be 
retrieved at the receiver. This concept is known as subcarrier 
modulation (SCM) and was described in the wireless infrared 
communications context (TJ Ch. 5]. Therefore, the power 
efficiency compared to M-PAM can be improved since SCM 
allows the use of power-efficient higher-order modulation 
formats with IM/DD systems. In O, the SCM concept 
was experimentally demonstrated, and in fT5l and fT6l , a 
novel transmitter design for subcarrier quadrature phase- shift 
keying (QPSK) and 16-ary quadrature amplitude modulation 
(16-QAM) was presented. As for the conventional electrical 
channel, many subcarriers can be superimposed resulting in 
a frequency division-multiplexing (FDM) system, referred 
to as multiple- subcarrier modulation (MSM) in the wireless 
infrared context (TJ p. 122], and orthogonal frequency-division 
multiplexing (OFDM) if the carriers are orthogonal ifTTl . 
Further, a subclass of OFDM known as discrete multitone 
(DMT), where the output of the inverse fast Fourier transform 
modulator is real instead of complex, was investigated in fT8l . 
In IU Sec. 5.3.2] and (19), MSM was shown to have poor 
power efficiency compared to single- subcarrier modulation. 
Specifically, Barry Q] Sec. 5.3.2] showed that the bandwidth 
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of a multiple- subcarrier system is independent of the number 
of subcarriers, and that the average optical power penalty for 
multiple- subcarrier M-QAM in comparison with the single- 
subcarrier case is 5 log 10 N dB, where N is the number of 
subcarriers. In [20], DMT was shown to suffer considerably 
in peak-power limited systems. Kang and Hranilovic ifTTTl and 
You and Kahn [22] proposed techniques to reduce the MSM 
power penalty. 

For single- subcarrier modulation formats, one option is 
that the DC bias required to ensure the nonnegativity of the 
electrical waveform does not carry information JT| Ch. 5], [[141 . 
lfT6l . The second option is by allowing the DC bias to carry 
information, thus potentially improving the power efficiency. 
This was studied by varying the DC bias on a symbol-by- 
symbol basis in [23] and within the symbol interval in l22l . By 
guaranteeing nonnegativity, the investigation of lattice codes 
for IM/DD with AWGN became feasible Q, El. Lattice 
codes, which are finite sets of points selected out of an N- 
dimensional lattice, have been extensively used in the con- 
struction of higher-order modulation formats for AWGN chan- 
nels with coherent detection [|25l - l27l . In addition, techniques 
such as constellation shaping and nonequiprobable signaling 
have been used to further minimize the average power ll26lL 
EH). In 0, a signal space model for optical IM/DD channels 
was presented, where average and peak optical power were 
considered as design constraints for constructing lattice-based 
modulation formats. Moreover, constellation shaping to reduce 
the average optical power were studied in ll24l for the case 
where no amplification is used, and in [[291 where optical 
amplifiers are used. An interesting question is if there exist 
new constellations that perform better than already known 
ones, with or without coding. 

In this work, we address this question by optimizing single- 
subcarrier modulation formats for uncoded IM/DD systems, 
with and without confining them to a lattice structure. We 
choose a uniform probability distribution over constellation 
points as in Q-El, Q, E3, ED, (23|, E1-G2. We propose 
a set of 4-, 8-, and 16-level single- subcarrier modulation 
formats which are optimized for average electrical, average 
optical, and peak power. These optimization criteria are all 
relevant, because the average electrical power is the standard 
power measure in digital and wireless communications [33l 
p. 40] and it helps in assessing the power consumption 
in optical communications [34], while the average optical 
power is an important figure of merit for skin- and eye- safety 
measures in wireless optical links E] Ch. 5], £), and 
helps in quantifying the impact of shot noise in fiber-optical 
communications EH p. 20]. In addition, the peak power, 
whether electrical or optical, is relevant for investigations of 
tolerance against the nonlinearities present in the system ll36l . 
We then analyze the performance of the obtained modulation 
formats in terms of mutual information at different signal-to- 
noise ratios (SNR), in order to predict their performance in the 
presence of capacity-achieving error-correcting codes. Finally, 
we optimize modulation formats analytically in the wideband 
regime, i.e., at low SNR, while assuming uniform probability 
distributions and compare them with other formats. 

The remainder of this paper is organized as follows. Sec- 
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Fig. 1: (a) Baseband transceiver with constrained-input Gaus- 
sian channel, (b) Passband transceiver of IM/DD systems. 



tion [III presents the system model. Section Hill elaborates on the 
signal space model, the performance measures, and the single- 
subcarrier modulation family which is the focus of this work. 
Section [TV] explains the optimization criteria that were used 
and describes the obtained modulation formats. In Section IVl 
we evaluate the performance of these modulation formats 
in the absence and presence of capacity-achieving error- 
correcting codes and we compare them with already known 
modulation formats. In addition, we analytically optimize 
modulation formats for the low-SNR regime. In Section |VH 
we summarize the main results and conclusions of this work. 

II. System Model 

The system model under study is depicted in Fig. [Ha). 
It consists of a modulator which maps the symbol u(k) 
at instant to a waveform belonging to the signaling set 
S = {so(t),si(t), . . . ,SM-i(t)}, where T s is the symbol 
period, M is the size of the signaling set, and si(t) = for 
t £ [0, T 8 ) where i = 0, 1, . . . , M - 1. For the time-disjoint 
signaling case, the generated waveform 



x(t) = ^2 s u(k)(t ~ kT s ) : 



(1) 



where u(k) is an ergodic process uniformly distributed over 
{0,1,...,M— 1}, is constrained to being real and nonnegative. 
The received signal can be written as 



y{t) = x{t)+n{t), 



(2) 



where n(t) is a zero-mean Gaussian process with double-sided 
power spectral density No/2. It should be noted that there 
exists no nonnegativity constraint on the signal y(t). This 
is then followed by the demodulation of y(t) which yields 
u(k), an estimate of u(k). The demodulator is a correlator or 
matched filter receiver, which minimizes the symbol error rate 
at a given SNR [33, Sec. 4.1]. This model is different from the 
conventional AWGN channel by the fact that the input x(t) is 
constrained to being nonnegative. 

The baseband model in Fig. [TJa) has been extensively 
studied in the optical communications context, since it serves 
as a good model for IM/DD systems d Ch. 5], Q, 0- 
@, G3 Sec. 11.2.3]. The passband transceiver for IM/DD 
systems is depicted in Fig. \Hb). In such systems, the elec- 
trical nonnegative waveform x(t) directly modulates a light 
source, such as a laser diode. Therefore, the information 
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is carried on the envelope of the passband signal z(t) = 
^y2cx(t) cos(2tt f Q t + 0(t)), i.e., the intensity of the optical 
field, where c represents the electro-optical conversion factor 
in watts per ampere (W/A) 051. ESI. l39i /*„ is the optical 
carrier frequency, and 6(t) is a random phase, uniformly 
distributed in [0, 2tt) and slowly varying with t. It then 
propagates through the optical medium depicted as an optical 
fiber in Fig. \V[b), which could be a free- space optical link in 
other applications. At the receiver, the photodetector detects 
the power of z(t). Since the dominant channel impairment in 
optical IM/DD systems is the thermal noise resulting from the 
optical-to-electrical conversion |[29lL ll40l p. 155], the received 
electrical signal can be written as 



y(t) = rcx(t) + n(t), 



(3) 



where r is the responsivity of the opto-electrical converter 
in AAV. Without loss of generality, we set rc = 1, which 
yields ©. 

There exists another IM/DD model which is relevant when 
the dominating noise comes from optical amplifiers, and not 
the receiver EZl Sec. 11.2], ED, E2- The noise in that 
model has a noncentral x 2 -distribution and not a Gaussian 
distribution as in this work. 

III. Signal Space Model 

By defining a set of orthonormal basis functions fa (t) for 
k = 1, 2, . . . , N and TV < M as in 0, each of the signals in 
S can be represented as 



N 



S i( t ) = y^ s i,kfa(t) 



(4) 



k=l 



for i = 0, ...,M- 1, where s« = (s»,i, s i>2 , - . - , s^jv) 
is the vector representation of Si(t) with respect to the 
aforementioned basis functions. Therefore, the constellation 
representing the signaling set S can be written as ^ = 
{so, si, . . . , sm-i}- With this representation, the continuous- 
time channel models in © and © can be represented by the 
discrete-time vector model 



y(fe)=x(fe)+n(fc), 



(5) 



where, at instant k, x(fc) G Q is the transmitted vector and 
n(fc) is a Gaussian random vector with independent elements, 
zero mean, and variance A^o/2 per dimension. Since x(fc) and 
y(fc) are both stationary processes, the argument k will be 
dropped from now on. To satisfy the nonnegativity constraint 
of the channel, the basis function fa(t) is set as in El, H to 



,1 ft 
'Jiyt) = \ — rect — 



(6) 



where 



rect(t) 



1, if < t < 1 
0, otherwise. 



This basis function represents the DC bias. Thus, s^i is chosen 
for each i = 0, . . . , M — 1 such that 

mm Si (t) > 0, 



which guarantees the nonnegativity of x(t) in (QJ. The admis- 
sible region T containing the set of all signal vectors satisfying 
the nonnegativity constraint can be represented as (71 Eq. (10)] 

N 

T = {w G R N : min V w k fa(t) > 0}, (7) 

where w = (wi,W2, • • • , wn). Therefore, the constellation O 
is a finite subset of T. The admissible region T for IM/DD 
systems has been shown in [7, Th. 1] to be the convex hull 
of a generalized TV-dimensional cone with vertex at the origin 
and opening in the dimension spanned by fa(t). 

A. Performance Measures 

Unlike the conventional electrical AWGN channel where the 
two standard power performance measures are the average and 
peak electrical power, three important performance measures 
for IM/DD channels can be extracted from the baseband and 
passband models in Fig. Q] The first entity is the average 
electrical power defined as 

1 f T 

P e = lim — / x 2 (t)dt, 
e T^oo 2T J_ T v J ' 

which for any basis functions can be simplified to 



Pe = ^ = ^E[|M| 2 ], 
-L s -L s 



(8) 



where E s is the average energy of the constellation, E[-] is 
the expected value, and / is a random variable uniformly 
distributed over {0, 1, . . . , M — 1}. This entity is an important 
figure of merit for assessing the performance of digital and 
wireless communication systems EH p. 40]. Therefore, it is 
relevant for IM/DD systems for compatibility with classical 
methods and results l42ll . l43l . In addition, it helps in quan- 
tifying the impact of relative intensity noise (RIN) in fiber- 
optical links l35l . and in assessing the power consumption of 
optical systems 1341 . In l30lh P e was used as a performance 
measure for comparing different intensity modulation formats. 

The second measure is the average optical power P Q , 
which has been studied in (D, 0, for the wireless 

optical channel. Limitations are set on P Q for skin- and eye- 
safety standards to be met. In fiber-optic communications, this 
entity is used to quantify the impact of shot noise on the 
performance (35l p. 20]. It is defined as 



Po 



lim — — 

T^oo 2T 



z 2 (t) dt 



lim — — 

T^oo 2T 



x(t) dt. 



This measure depends solely on the DC bias required to make 
the signals nonnegative and can be represented in terms of the 
symbol period and constellation geometry as 151. 171 

P = ^Le[s /?1 ], (9) 



regardless of fa(t), . . . ,4>N(t). 

The third measure is the peak optical power defined as 



max ■ 

t 



— cmaxx(t). 

t 



(10) 
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It is relevant for investigations of tolerance against the non- 
linear behavior of transmitting and receiving hardware in 
communication systems Q, (36), J39l and has been studied 
in (7), (HI, (30ll . The peak electrical power P e is directly related 
to P by 

A = & ' 



and will not be further considered in this paper, since a 
constellation optimized for P Q will automatically be optimized 
for P e too. A general form for P Q as a function of Q, as in ([8]) 
and © for P e and P G , does not exist, since P Q depends on the 
exact choice of basis functions. A special case will be studied 
in Sec. HITBl 

To assess the performance of the different modulation 
formats in the presence of capacity-achieving error-correcting 
codes, we consider the mutual information EU Sec. 2.4] 



J(x;y)=tf(x)-tf(x|y) 



(11) 



as a performance measure. The terms H(x) and il(x|y) are 
the entropy of x and the conditional entropy of x given the 
received vector y, averaged over both x and y. The channel 
capacity of a discrete memory less channel is [44 1 Eq. (7.1)] 



C 



max/(x; y), 

p(x) 



(12) 



where the maximum is taken over all possible input dis- 
tributions p(x). For a fixed constellation and distribution, 
the mutual information gives a lower bound on the channel 
capacity. In our work, like in the works of many other authors, 
we choose a uniform distribution over the constellation points. 

We define R s = 1/T S as the symbol rate in symbols per 
second, = R S R as the bit rate in bits per second, and = 
E s I R as the average energy per bit. Furthermore, in order to 
have a fair comparison of the bit rates that can be achieved 
by the different modulation formats in a fixed bandwidth, the 
spectral efficiency defined as 

Rb 



v 



w 



[bit/s/Hz] 



should be taken into account, where W is the baseband 
bandwidth defined as the first null in the spectrum of x(t). In 
this paper, we are interested in two extreme cases: the uncoded 
system, for which R = log 2 M, and the system with optimal 
coding, for which R = /(x; y). 

B. Single -Sub carrier Modulation Formats 

For in-phase and quadrature phase (I/Q) modulation formats 
to be used on intensity modulated channels, a DC bias is 
required in order for x(t) to be nonnegative. This could be 
translated geometrically by having a three-dimensional (3d) 
Euclidean space spanned by the orthonormal basis functions 
fa (t) defined in © and 



iW = 



2 ft 
— cos (2tt/£) rect f — 



(13) 



n{t) = x/^r sin(27r/t) rect ( ±. ) , 



which are the basis functions of conventional I/Q modula- 
tion formats such as M-PSK and M-QAM, where / is the 
electrical subcarrier frequency d, (7). As in (TJ pp. 1 15— 
116] and Q, we use / = 1/T S , which is the minimum 
value for which fait), fait), and fait) are orthonormal. 
In 0, IM/DD modulation formats based on these three basis 
functions are referred to as raised-QAM, and in iTUfll as single 
cycle SCM. At the same symbol rate, modulation formats 
such as OOK and M-PAM have W = R s , whereas the 
modulation formats belonging to the single- subcarrier family 
occupy W = 2R S ; this is due to the intermediate step of 
modulating the information onto an electrical subcarrier before 
modulating the optical carrier [Q] Ch. 5], (3Ql . 

We now describe explicitly the admissible region T for 
single- subcarrier modulation formats [3, Fig. 4.2], Q. 

Theorem 1: For the specific set of basis functions fait), 
fait), and fa{t) defined in ©, (fT3l) , and (fl4l) . the admissible 
region T is a three-dimensional (3d) cone with vertex at the 
origin, apex angle of cos -1 (1/3) = 70.528°, and opening in 
the dimension spanned by fa (£) . 

Proof: The admissible region in © can be written for 
single- subcarrier modulation formats as 



T = {wGM 3 : min ^ w k </) k (t) > }, (15) 



where 



min y^w k fait) 



1 



{wi 



+ y 2{wl + wfj cos (2tt/J -0)} 
= -^( Wl -yj2(w*+w*)), (16) 

V J- s 

where 6 = arg(^2 +^3). Therefore, substituting (IT6b in dT31) 
yields 



T {w e M 3 : w 1 > 



2iw\ 



(17) 



which is a 3d-cone with apex angle of cos -1 (1/3) = 70.528° 
pointing in the dimension spanned by fa (t), with vertex at the 
origin. ■ 
The average electrical and optical power were given in {8]) 
and © as functions of the constellation ft; however, the peak 
optical power defined in (TTTTb could for these basis functions 
be expressed in terms of the constellation geometry too 
Fig. 4.3]. 

Theorem 2: The peak optical power for the single- 
subcarrier modulation formats with the above defined basis 
functions can be expressed as 



Pn = 



max 



(18) 
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Fig. 2: (left to right): Contours of equal P e , P Q , and P Q 

Proof: From (ITUb . P can be written as 

P Q = c max Si (t) 

i,t 

3 

= cmaxY^Si fc0fc(*) 



k=l 



:{s»,i + ^2(s? 2 + s? 3 ) }. 



Alternatively, the theorem can be proved using (7J Th. 2]. ■ 

IV. CONSTELLATION OPTIMIZATION 

To design power-efficient constellations, the admissible re- 
gion in (ITTl) has to be taken into account. As done before for 
the conventional AWGN channel I45l - ll49lh our approach of 
finding the best constellations can be formulated as a sphere- 
packing problem with the objective of minimizing a cost 
function depending on the constraints that might be present 
in the system model shown in Fig. Q] Thus, the optimization 
problem, for given constants M and d m { n , can be written as 



Minimize 
Subject to 



\n\ = m 
ft cr 

d(ft) = d n 



(19) 
(20) 
(21) 
(22) 



where 



d(fi) 



mm | 



Choosing the objective function as = E[||s/|| 2 ] results 
in ft = c tfp ej M^ Le -' a constellation optimized for average elec- 
trical power, and £(Q) = E[sj 5 i] results in ft = c &p o ^m-> Le -' a 
constellation optimized for average optical power. Finally, 



max 



{*M + ^2(4 2 + 4 3 ) } 



yields ft = c £p M , a constellation optimized for peak optical 
power. Fig. [2] depicts a two-dimensional contour plot of the 
three objective functions together with the admissible region 
T. The constraint in (BTt guarantees that the signals belong to 
the admissible region T, therefore satisfying the nonnegativity 
criterion of the channel. The minimum distance d m [ n in (l22b 
serves as a good measure of error probability performance 
in the presence of AWGN at high SNR. Although this op- 
timization problem is well formulated mathematically, it is 
difficult to obtain an analytical solution. Therefore, we resorted 
to numerical optimization techniques as in l45ll - |49l to find 



the best constellations. One drawback of such constellations 
is often the lack of geometric regularity, which increases 
the modulator and demodulator complexity. The optimization 
problem is nonconvex; therefore, a local solution does not 
imply that it is globally optimal. 

A special case of this optimization problem, which might 
not guarantee the optimal solution, is to confine the possible 
constellations to have a regular structure such as that of a 
lattice, denoted by A. In this case, the above optimization prob- 
lem can be reformulated by replacing (f2TI) with ft C T D A, 
and dropping (l22l) since it is directly inferred by ft C A. 
This was done in for the cubic and Leech lattices. In 
order to compare with the best nonlattice constellations of 
relatively small sizes M, we use in this work the face-centered 
cubic lattice (A3), which provides the densest packing for 
the 3d-Euclidean space (27l p. xvi]. The obtained lattice- 
based constellations optimized for average electrical, average 
optical, and peak optical power are denoted J2fp e5 M> ^p ,m> 
and Jzfp M , respectively. 

A. Optimized Constellations 

In Figs. [3H3 the results of the numerical optimizations are 
illustrated for M = 4, 8, and 16, for unconstrained sphere 
packings (^) and lattice codes (jSf ), and for the three power 
measures. Their coordinates are included in App. El It should 
be noted that rotations of all constellations about <p\ (t) do not 
change the power requirements. As we shall see in Sec|Vl the 
obtained constellation outperform previously known formats. 

Conjecture 3: All constellations in App. El are optimal 
solutions of (IT9W221). 

1) A-level Constellations: The same 4-level constellation 
provides the lowest P e , P Q , and P Q while satisfying the 
optimization constraints. The geometry of this constellation is 
a regular tetrahedron where all the spheres, or the constellation 
points lying at the vertices of this regular tetrahedron, are 
equidistant from each other. This constellation is also the result 
of the optimization constrained to ft C T D A3, where the 
apex of the cone coincides with a point in the As lattice and 
the lattice is oriented such that two lattice basis vectors lie 
in the plane spanned by <j>2(f) and 03 (t). Since the obtained 
constellation is optimized for P e , P Q , and P Q , we will refer to 
it as ^4 or ^4. 

It is a remarkable fact that the vertex angle of the tetra- 
hedron, defined as the apex angle of the circumscribed cone, 
is exactly cos -1 (1/3), which is equal to the apex angle of 
the admissible region T. Thus, ^4 fits T snugly, in the 
sense that all constellation points are equidistant from each 
other and lie on the boundary of T. For constellation points 
regarded as unit-diameter spheres, ^4 can be illustrated as 
four spheres touching each other and the boundary of a larger 
cone as shown in Fig. [3 This, as we shall see in the next 
section, makes the modulation format very power-efficient. 
This modulation format consists of a zero-level signal and a 
biased ternary PSK constellation l50lh |5H . In prior work l30lh 
the ^4 format was introduced where it was called on-off 
phase-shift keying (OOPSK), and in |3H . it was demonstrated 
experimentally. Other hybrids between amplitude- shift keying 
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and PSK have been studied in l32l and l52| ; however, such 
modulation formats do not satisfy the nonnegativity constraint 
of IM/DD channels. 

2) 8-level Constellations: The highly symmetric and com- 
pact constellation ^p & 8 consists of four central spheres ar- 
ranged in a tetrahedron and four additional spheres, each 
touching three spheres in the central tetrahedron. Surprisingly, 
seven of the eight spheres touch the conical boundary of T. 
This modulation format is a hybrid between 2-PAM and two 
ternary PSK constellations, which are DC-biased differently. 
The constellation c €p o 8 is the same as c £p e 8 but with the 
top central sphere moved to the boundary of the admissible 
region. The constellation optimized for peak optical power, 
8 , consists of two tetrahedra lying on top of each other, 
where one is reflected and rotated 7r/3 about <fri(i). 



On the other hand, when confining the set of points to a 
lattice structure, the resulting constellations which provide the 
lowest P e and P Q are the same, £?p e ^ = ^p G ,8- However, the 
lattice-based constellation which is optimized for P Q , J£p 8 , 
is the same as ^p g . 

3) 16-level Constellations: The constellations ^p e iQ, 
c tfp o 16 , and ffp 16 are not lattice codes; however, the shape 
obtained could be well justified by the contour plots shown 
in Fig. [2 The c €p o 16 constellation contains the constellations 
c ^ > P o 8 an d ^p e ,8' whereas ^p 16 consists of two tetrahedra 
of which one is reflected and eight spheres lying in between, 
almost at the same level. The presence of two tetrahedra in 
the constellations optimized for peak optical power, whether 
8- or 16-levels, is due to the fact that these constellations are 
bounded from above by another cone with the same apex angle 
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as that of T. This can be inferred from the expression of P Q 
in flU) Th. 2]. 

However, when only lattice-based structures are considered, 
the constellations providing the lowest P e , P Q , P Q are the same, 



i.e., 



°^16 — ^ ) 



P e ,16 



= J2f i 



P G ,16 



Po,16- 



it can be noticed that Jzfi6 contains both 8 and ££p 



From Figs. 0]-[5j 



and that the % constellation is included in all the obtained 
constellations. 

Conjecture 4: For single- subcarrier IM/DD systems, the ^4 
constellation is included in all optimal constellations with 

M > 4. 

B. Previously Known Constellations 

Our investigation encompasses some previously best known 
formats, which are presented after being normalized to unit 
dmin. Readers are referred to ll53l for the performance of 
classical formats such as M-PSK and M-QAM over IM/DD 
channels. At spectral efficiency 77 = 1 bit/s/Hz (where R = 
log 2 M), OOK is defined as {(0), (1)} in terms of 0i(t). 

At spectral efficiency 77 = 1.5 bit/s/Hz, a star- shaped 
8-QAM E3 denoted as 8-QAM, in which the DC bias 
is allowed to vary from symbol to symbol, is defined 
as {(1, ±1/2, ±1/2), ((1 ± y/3)/y/2, 0, ±(1 ± y/S)/2), ((1 ± 
V3)/y/2,±(l ± v / 3)/2,0)}. At spectral efficiency 77 = 2 
bit/s/Hz, nonnegative 4-PAM is defined as {(0), (1), (2), (3)} 
in terms of (f>i(t), and a version of 16-QAM denoted 
as 16-QAM where the DC bias varies from symbol to 
symbol, is defined as {(1, ±1/2, ±1/2), (y/E, ±1/2, ±3/2), 
(y/E, ±3/2, ±1/2), (3, ±3/2, ±3/2)}. 

V. Performance Analysis 

In this section, we assess the performance of all the mod- 
ulation formats considered in this work in terms of their 
symbol error rate (SER) performance, asymptotic power gain 
versus OOK, which was used as a benchmark in d, EL 
and spectral efficiency. Sections IV-AHV-Bl consider uncoded 
transmission (R = log 2 M) and Sections IV-CHV-Dl consider 
coded transmission (R = /(x; y)). 

A. Symbol Error Rate 

For the 4-level modulation ^4 presented in Fig. [3 deriving 
the exact theoretical SER is not straight-forward, due to the 
irregularity of the Voronoi regions. However, it has the same 
structure as the simplex signal set in [33] Sec. 4.1], although 
it is DC-biased to be used in IM/DD systems. The exact SER 
of an M-ary simplex signal set is [|33j Eq. (4.116)] 



1- QU 



2E 



s, simplex 



N 

-u 2 /2 



M 



M-l 



2tt 



du, 



(23) 



where E Sy simplex is the average symbol energy of the zero-mean 
simplex constellation and Q(x) = \jyp2m J^°°exp(— u 2 /2) du 
is the Gaussian Q-function. For M = 4, the energy of the 
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Fig. 6: Simulated (solid) and theoretical (dotted) SER for 
the modulation formats vs. jE b (top), jp o (middle), and jp 
(bottom) without coding. 
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zero-mean simplex constellation is E S: simplex = E s /2 = E^, 
where E s and E b are the average symbol and bit energies, 
respectively, of the ^4 constellation. Hence, the exact SER of 

^4 is 

3 



1 



1 




72 



du. (24) 



For higher-level modulation formats, the standard union 
bound found in [|33l Eq. (4.81)] is used to approximate the 
theoretical SER. This union bound can be approximated as 



M ^ 



1 d 2 - 

2Nn 



where K is the number of distinct signal pairs (si(t),Sj(t)) 
with i < j for which f(si(i) - Sj(t)) 2 dt = d min . This 
approximation is tight at high SNR. 

Fig. [6] (top) shows the simulated and theoretical SER of the 
studied modulation formats vs. electrical SNR defined as 

101og 10 # [dB]. 



No 



(26) 



As expected, all the modulation formats which are optimized 
for P e outperform the other formats at the same spectral 
efficiency. For spectral efficiency 77 = 1 bit/s/Hz (blue), ^4 
has a 0.86 dB average electrical power gain over OOK to 
achieve P s = 10 -6 . For 77 = 1.5 bit/s/Hz (red), c £p e 8 has a 
0.31 dB gain over c € Pq >8 , 0.58 dB gain over Jfp^lW dB 
gain over 8 , and 2.55 dB gain over 8-QAM to achieve 
P s = 10- 6 . For 77 = 2 bit/s/Hz (black), ^ 16 has a 0.15 
dB gain over c €p o \§, 0.47 dB gain over 16 , 0.74 dB gain 
over «5fi6, 2.65 dB gain over 4-PAM, and 2.80 dB gain over 
16-QAM. The modulation formats optimized for P e and P Q 
are close in performance to those optimized for P Q and to the 
lattice-based formats. 

In order to compare modulation formats in terms of their 
average optical power requirements, we define the average 
optical SNR as 



7p o = 10 log! 



Po 



[dB] 



(27) 



in a similar fashion as in ^ Eq. (5)]. Using ([8]), (|9]), (l26i 

rc = 1, and E s = E b RbT s , this expression can be written as 



7p g 



1 



<y Eb + 10 log 



10 



(28) 



where the first term depends on the regular (electrical) SNR 
and the second depends only on the constellation geometry. 
Fig. [6] (middle) shows the SER plotted vs. jp o . Quite obvi- 
ously, the modulation formats optimized for P Q perform better 
than the rest. For 77 = 1 bit/s/Hz, ^4 has a 0.43 dB average 
optical power gain over OOK to achieve an SER of 10 -6 . For 
77 = 1.5 bit/s/Hz, tf Po 8 has a 0.04 dB gain over tfp e 8 , 0.46 dB 
gain over J£p e 8 , 0.84 dB gain over ffp 8 , and 1.35 dB gain 
over 8-QAM to achieve P s = 10 -6 . For 77 = 2 bit/s/Hz, &p oil6 
has a 0.13 dB gain over c tfp e 16 , 0.45 dB gain over *€p 16 , 0.66 
dB gain over JSf 16 , 1.36 dB gain over 16-QAM, and 1.44 dB 
gain over 4-PAM to achieve P s = 10 -6 . It can be noticed that 



the 8 -level modulation formats optimized for average electrical 
and optical power are very close in performance. 

In a similar fashion as jp o , we define the peak optical SNR 



7p o = 10 log 10 



[dB] 



(29) 



in order to assess the different modulation formats under 
study in terms of their peak optical power requirements. 
Using ©, CES), rc = 1, and E s = E b R b T s , this 

expression can be written as 



(25) 7p o = ^7^ + 10 log 



max? 



{*,i + \f 2 ( s h + 4 3 ) } 



10 ■ 



(30) 



where the first term depends on the regular (electrical) SNR 
and the second depends only on the constellation geometry. 
Fig. [6] (bottom) shows the SER plotted vs. jp . It is clear 
that the modulation formats optimized for peak optical power 
outperform the other formats. For 77 = 1 bit/s/Hz, OOK has 
a 0.82 dB peak optical power gain over ^4 to achieve P s = 
10~ 6 . For 77 = 1.5 bit/s/Hz, ^ 8 has a 0.46 dB gain over 
^P e 8 , 1.20 dB gain over c tfp o 8 , 1.25 dB gain over ^fp e 8 , and 
1.72 dB gain over 8-QAM to achieve an SER of 10" 6 . For 
77 = 2 bit/s/Hz, ffp 16 has a 0.81 dB peak optical power gain 
over Jzfi 6 , 0.98 dB gain over 4-PAM, 1.00 dB gain over tfp eA6 , 
1.16 dB gain over ffp oil6 , and 2.42 dB gain over 16-QAM to 
achieve P s = 10 -6 . Overall, modulation formats optimized for 
P perform well in average-power limited systems. In l54l . a 
similar conclusion was reached for the case of coherent optical 
systems. 



B. Asymptotic Power Efficiency 

At asymptotically high SNR, the performance difference 
in dB between the different modulation formats approaches 
constant values. We use OOK as a benchmark for power- 
efficiency on IM/DD channels as in (J), O. 

Fig. [7] (top) presents the average electrical power gain 



= 10 log 10 



e,OOK 



[dB] 



(31) 



of a modulation format in comparison to OOK in order to 
achieve the same error rate performance at asymptotically high 
SNR, where P e ,ooK and P e are the average electrical power 
of OOK and the modulation format under study, respectively. 
For spectral efficiency 77 = 1 bit/s/Hz, ^4 has a 1.25 dB 
average electrical power gain over OOK. The overall trend 
at asymptotically high SNR is similar to the comparison of 
average electrical SNR 7^ in Sec. IV- Al to achieve P s = 10 -6 . 
For 77 = 1.5 bit/s/Hz, the 8-level modulation format optimized 
for P e , i.e., ^p e 8 , has the smallest average electrical power 
penalty of 0.67 dB compared to OOK. The asymptotic average 
electrical power gain of ^p & 8 over other formats is larger 
than the gains at P s = 10 -6 with the exception of having a 
similar gain when compared to ££p & 8 and *€p g where both 
are lattice-based formats. For 77 = 2 bit/s/Hz, ^p e i 6 has a 
2.42 dB penalty with respect to OOK at asymptotically high 
SNR. Its asymptotic electrical power gain is larger than the 



KAROUT et al: OPTIMIZING CONSTELLATIONS FOR SINGLE-SUB CARRIER INTENSITY-MODULATED OPTICAL SYSTEMS 



9 



,_, -1 - 



°5-1.5 



( 6 ry 



8-QAM 



16-QAM 



r] [bit/s/Hz] 



© OOK 



> ~^Pe,8 

8-QAM + 



* W P ,16 
^P e ,16 Q 

„ %,,16 
^16 > 



16-QAM* 4-PAM 



1.6 

rj [bit/s/Hz] 



O OOK 
□ ^4 



^F 0) 8 

n ^P e ,8 



^P G ,16 



> ^16 

~ a ^P e ,i6 
®P ,16* 4-PAM 



3-QAM 



16-QAM ■+ 



1.6 

T] [bit/s/Hz] 



Fig. 7: Average electrical (top), average optical (middle), and 
peak optical (bottom) power gain of the various modulation 
formats vs. OOK without coding. 



gains at P s = 10 -6 except when compared to J£fi6 where the 
gain at asymptotically high SNR is smaller by 0.08 dB. 
Fig. [7] (middle) shows the average optical power gain 

^o,OOK 



gam 



10 log 



10 ' 



Po 



[dB] 



(32) 



of the modulation formats under study with respect to OOK 
at asymptotically high SNR, plotted versus their spectral 
efficiencies. The average optical power of OOK is P ,ook, 
whereas the average optical power of the modulation format 
under study is denoted as P Q . For r] = 1 bit/s/Hz, ^4 offers 
0.62 dB average optical power gain over OOK. The asymptotic 
average optical power gain of ^4 over OOK is larger than 
the gain at P s = 10" 6 in Sec. FT-Al For 77 = 1.5 bit/s/Hz, 
c top o 8 has a 0.71 dB average optical power penalty compared 
to OOK to achieve the same error rate at asymptotically high 
SNR. Furthermore, the asymptotic average optical power gains 
of ^p o? 8 are larger than the gains when compared with other 
formats to achieve P s = 10 -6 . Exceptions are when compared 



to J% 



and 



where the gains are smaller. Another 



observation is that the performance of c £p o 8 becomes similar 
to that of ^p e>8 . For rj = 2 bit/s/Hz, ^T 16 has a 1.65 dB 
penalty with respect to OOK at asymptotically high SNR. 
Compared to the average optical SNR performance gains at 
P s = 10 -6 , the asymptotic gains are larger except the gain 
over i6 and 

If peak power is the limiting factor in a communication 
system, Fig. [7] (bottom) shows the performance of the various 
modulation formats with respect to their peak optical power 
gain with respect to OOK 



10 log 



>,OOK 



10 ' 



[dB], 



(33) 



where P ,ook and P Q denote the peak optical power of OOK 
and the modulation format under study, respectively. For 77 = 1 
bit/s/Hz, OOK has the best performance. It has a peak optical 
power gain of 0.62 dB over ^4. The asymptotic peak optical 
power gain of OOK compared to ^4 is less than the gain 
in peak optical SNR to achieve P s = 10 -6 . In other words, 
the peak optical performance of ^4 gets closer to OOK at 
asymptotically high SNR. For 77 = 1.5 bit/s/Hz, g has the 
smallest penalty of 1.51 dB compared to OOK. The asymptotic 
peak optical power gain of 8 over other formats is larger 
than the gain reported in Sec. IV-A[ The exception is the 



when compared to c £p e 8 and ££ 1 



performance of % 
where the asymptotic peak optical power gain is similar to 
that at P s = 10 -6 . For 77 = 2 bit/s/Hz, *€p 16 outperforms 
the other modulation formats with a penalty of 2.13 dB with 
respect to OOK at asymptotically high SNR. Even though the 

over most other 



asymptotic peak optical power gain of ^ 
formats is larger than that at P s = 10" 
ffp 16 is compared to ^p e ,i6 ? and <?p 05 i6 
differences between the gains to achieve P s = 10 -6 and the 
gains at asymptotically high SNR did not change the order of 
which formats have a better performance. However, 4-PAM at 
asymptotically high SNR is now worse than ^p e5 i6 and ^p 05 i6 
in terms of peak optical power. Finally, the trend that can be 
observed in Fig. [7] (bottom) is that the gap in performance 



16 

it is smaller when 
. So far, all the 
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between the modulation formats optimized for peak optical 
power and the rest of the formats under study gets larger with 
higher spectral efficiencies. 

C. Mutual Information vs. SNR 

Fig. [8] depicts the spectral efficiency where R = /(x; y) 
of the modulation formats vs. the different SNR measures, 
i.e., average electrical, average optical, and peak optical SNR. 
We study the same modulation formats as before, designed to 
minimize the uncoded SER, although they are now evaluated 
in a coding context. The mutual information /(x; y) for a 
fixed constellation and input distribution gives a lower bound 
on the rate in bits per symbol at which information can be sent 
with arbitrarily low probability of error, i.e., with the use of 
optimal coding. Throughout this paper, we consider uniform 
input distributions. 

In Fig. [8] (top), the spectral efficiency is plotted vs. jE b - 
The best format in terms of jE b is ^4 for < 77 < 0.74, 
#p o>8 for 0.74 < 77 < 0.99, and #p o>16 for 77 > 0.99. It is 
surprising that the formats optimized for P Q outperform the 
formats optimized for P e , when compared in terms of 7# b . 
This is different from the uncoded case in Sec. IV-AHV-Bl The 
8-QAM and £?p ej s formats have a better performance than 
ffp g . Other observations are that ^p 16 outperforms Jzfi6, 
and 4-PAM has the lowest performance. 

In a similar fashion as above, Fig. [8] (middle) shows the 
spectral efficiency 77 plotted vs. the SNR jp o for the same 
modulation formats. The best format is ^4 for < 77 < 0.83, 
#p o>8 for 0.83 < 77 < 1.18, and &p oil6 for 77 > 1.18. 
As opposed to the performance vs. 7# b , the gap between 
the formats optimized for average optical power and those 
for average electrical power is larger. It is interesting that 
the modulation formats optimized for average optical power 
perform better than the rest for systems which are limited by 
either average electrical or optical power. 

Fig. [8] (bottom) presents the spectral efficiency vs. jp . But 
here, the story is a bit different. The best format is OOK 
for < 77 < 0.93, and 16 above that. An interesting 
observation is that 4-PAM performs better than ^p 16 for 
< 77 < 0.56. Furthermore, modulation formats optimized for 
peak power perform better in average-power limited systems 
than when using formats optimized for average power in peak- 
power limited systems. 

D. Mutual Information in the Wideband Regime 

It is apparent from the results in Sec. lV-Cl that constellations 
optimized for a minimum-distance criterion (i.e., uncoded 
transmission at high SNR) do not necessarily perform well in 
terms of mutual information, particularly not in the wideband 
regime (low spectral efficiency). We define the zero-crossing 
v(CL) of a constellation Q in terms of an SNR measure as the 
minimum SNR for which 77 > 0. For the three SNR measures 
considered in this work (77^, 7p o , or 7^ ), the zero-crossings 
will be denoted as z/# b (fi), z/p o (0), and vp (O), respectively. 
Fig. [U shows that in terms of jE b and 7p o , ^4 and OOK 
have the same lowest zero-crossing of 1.42 dB and —0.79 
dB, respectively, among all studied formats. However, OOK 
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has the lowest zero-crossing of 2.21 dB in terms of jp . The 
zero-crossings will be explained analytically in this section. 
An interesting question is if there exist other constellations 
that perform better in the wideband regime than those found 
in Sec. HV] As we shall see in this section, the answer is yes. 

Theorem 5: For any given M > 2 with \Q\ = M and a 
uniform input distribution, 

n lE^II 2 x_1 x 



I / Bt (fi) = 10Iog 10 1 



^E-=o IN 



l0g e 2 , 



Vp o (&) = 5 log 



10 



meZo 1 IN- 



EM—. 
2=0 




up (SI) 



5 log 



10 



V max; {s iA + ^2(«? 2 + «? 3 ) })' 



ME". 1 IN.II 2 



M-l 



log e 2 • 



Proof: According to (55l Th. 7], the zero-crossing 

1 



where 



u Eb (Q) = 101og 10 



a(fi) 




[dB], 



log 2 e, 



(36) 



(37) 



(38) 



and E[0] is the mean of the constellation. Substituting 
in ([37]) yields d34j), whereas substituting dH) in (EHJ and (El 
yields (l35t and (I36L respectively. ■ 
The problem of optimizing modulation formats which re- 
quire the least SNR for reliable communications can be 
formulated as finding the constellation Q which provide the 
minimum zero-crossing v(Q). It can be represented mathemat- 
ically as 

Minimize z/(ft) (39) 
Subject to \n\= M (40) 
OcT, (41) 

for the three SNR measures considered in this work. Observe 
that no minimum-distance condition applies. Using [55j Th. 7], 
this optimization problem can be solved analytically. 

Theorem 6: For any given M > 2 and OcT with \Q\ = 

M, 

i/ Sb (fi)>101og 10 ^^ [dB], (42) 

with equality if and only if M — 1 constellation points are at 
the origin. 

Proof: The numerator of (f34t can be bounded using 

2 M-l 

|2 



M-l 

2 = 



= E 

2=0 

M-l 

> E ii 

i=0 



M-2 M-l 

2 E E < s - s i) < 43 > 

2=0 j=i+l 



(44) 



where (•, •) for s^, Sj G T, denotes the inner product, and (l44t 
is due to the fact that 



(Si, Sj) — 



Is^H cos6> > 0, 



Vi,j = 0, ...,M — 1, since from Th. [T] max# = 
cos -1 (1/3) = 70.528° < 90°. Equality holds if and only if 
M — 1 points are located at the origin. Applying (l44t in ([34b 
completes the proof. ■ 

Several constellations fulfill Th. [6] If the nonzero constella- 
tion point has the coordinates (y/ME s , 0, 0), we refer to this 
constellation as $m- At the same symbol rate, $m occupies 
a bandwidth of W = R s , as OOK and M-PAM. However, 
if the nonzero constellation point is at the surface of T, this 
point has coordinates (y/(2/3)ME 8 ,0, ^ME s /3), and will 
be referred to as &m- The constellation @m belongs to the 
SCM family; therefore it occupies a bandwidth of W = 2R S , 
which is twice the bandwidth of OOK and M-PAM at the 
same symbol rate. The reason for choosing the two extremes 
S'm and Gm is that both have the same P e ; however, Gm has 
lower P Q (see Fig. [2]) and $m has lower bandwidth. 

Theorem 7: For any given M > 2 and OcT with |0| = 
M, 

v Po (fi) > 5 log 10 ^ 3(M 2 _ 1} log e 2^ [dB], (45) 

with equality if and only if O = Gm> 

Proof: By using ([35]), vp (O) can be bounded by 



M-l o 



i/p o (n) > 5iog 10 



> 5 log 10 



> 5 log 




log e 2 



(48) 



The inequality in (l46l) follows from s^ijSj,! > 0, Vi,j = 
0, 1,...,M - 1, whereas (07]) follows from (@4| and (l48l) 
follows from the definition of the admissible region in (fT71) . 
This completes the proof of (l45t . 

To prove that (l45l) is tight if and only if ft = Gm, we 
observe that (l46t and (1471) are tight if and only if M — 1 
points are at the origin, while (l48t is tight if and only if all 
M points are located on the boundary of the cone. 

■ 

So far, we were able to derive optimal constellations in 
terms of VE h and vp o . For the third SNR measure vp , 
however, we resort to a conjecture only, for which we have 
solid numerical support but no proof. 

Conjecture 8: For any OcT, 



>51og 10 (41og e 2), 



(49) 



with equality if and only if ft = $2 (OOK). 

Fig. [9] (top) shows the spectral efficiency in the low jE b 
regime for $m and Gm, where M = 4, 8, and 16. We also 
include some of the previously optimized constellations for 
comparison. Even though $m and Gm, for a given M, have 
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Fig. 9: Spectral efficiency of the optimal constellations §m an d 
0m in the wideband regime. Some of the previously optimized 
constellations are included for comparison. 



the same zero-crossing v Eh , Sm performs better at all nonzero 
spectral efficiencies. Indeed, d?ie, and g± are the best of 
all studied constellations for 77 < 0.70. As M — >• 00, <§m and 
<^m approach the coherent (non-IM/DD) Shannon limit, i.e., 
VE b {&M) = v Eb (0M) -1.59 dB. 

Fig. (middle) depicts the spectral efficiency in the low jp o 
regime. For M = 4, 8, and 16, the formats Gm have lower 
vp than §m\ however, the latter performs better than all 
other studied formats for 0.13 < rj < 0.80. Using (H5b . vp o (Vt) 
approaches — 00 as M —> 00 faster for ft = than <§m- 

Finally, we show the spectral efficiency in the low ^p 
regime in Fig. [9] (bottom). Surprisingly, OOK has the best 
performance up to 77 = 0.93, and then ^p 16 outperforms 
the rest. On the other hand, the modulation formats which 
minimize VE h (&) or vp (Q) do not have the best performance 
as before. The trend is that their performance is poorer with 
increased number of levels. Also, Gm is worse than $m 
for a given M, since the former has a higher peak power 
requirement due to the nonzero point on the surface of T. 

It may seem somewhat unexpected that a constellation with 
M — 1 points at the origin would perform better at low 
SNR than all other constellations. Such a constellation is 
equivalent to a binary constellation with a higher probability 
for the zero point. Similar constellations were found to achieve 
capacity in 1421 and l56l . albeit for different channels. A 
similar constellation was studied in l57l . where a signal set for 
coherent AWGN channels consisting of two antipodal signals 
and M — 2 signals at the origin was used to disprove the strong 
simplex conjecture, according to which the regular simplex 
signal set would minimize the uncoded probability of error 
under an average energy constraint. 

VI. Conclusions 

By using the minimum distance as a modulation design 
criterion for uncoded systems, we were able to numerically op- 
timize 4-, 8-, and 16-level single- subcarrier IM/DD modulation 
formats for systems which are limited by average electrical, 
average optical, and peak power. For M = 4, the most power- 
efficient modulation in terms of average electrical, optical, and 
peak power has a tetrahedral structure. This constellation is 
also a subset of all the obtained higher-level constellations. As 
for the 8- and 16-level constellations, power-efficient schemes 
are obtained by not confining the set of constellation points 
to a regular structure such as that of a lattice. However, this 
comes at the price of losing the geometric regularity, which 
increases the modulator and demodulator complexity. Our 
comparisons show that the penalty gap for using modulation 
formats optimized for P Q in systems which are P e or P Q 
limited is much less than the penalty gap when using formats 
optimized for P e or P Q in systems which are limited by P Q . 
Therefore, modulation formats optimized for P Q should be 
preferred in applications with mixed power requirements. The 
overall gain of the obtained formats over the previously best 
known formats is between 0.6 dB and 3 dB at asymptotically 
high SNR. We conjecture that the new obtained modulation 
formats are optimal for their size and optimization criteria over 
uncoded single- subcarrier IM/DD channels. 
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On the other hand, when capacity-achieving error-correcting 
codes are deployed, the best modulation formats in the wide- 
band regime (small rj) have only one nonzero constellation 
point. This is confirmed analytically and numerically. At 
higher 77, modulation formats optimized for P Q are able to 
achieve higher reliable transmission rates compared to other 
formats in systems which are limited by average electrical 
or optical power. OOK and the 16-level modulation format 
optimized for peak power offer the best performance in peak- 
power limited systems. The overall gain of the obtained 
formats over previously best known formats ranges between 
0.3 dB and 1 dB. 



Appendix A 
Obtained Constellations 

This appendix lists the coordinates of the numerically 
optimized constellations in Sec. IIV-A1 Whenever possible, 
numerical values have been replaced with the corresponding 
exact values. Constellations are normalized to unit d m [ n . 

cj? c/? _ c/?_ c/)_ c/)_ c/J 

— <^P e ,4 — ® 'P ,4 — ^P ,4 — ^PeA ~ ^PoA ~ PoA 

{(0. 0, 0), ( V273, 0, 1/V3), (72/3, ±1/2, - V3/6)}. 

^P e>8 =%U {((5/3)v^73, 0, -5/(3>/3)), 

((5/3) ±5/6, 5/(6V3)), (2^2/3, 0, 0)}. 

% 5 8 =%U {((5/3) v 7 ^, 0, -5/(3^3)), 
((5/3)v^73,±5/6,5/(6V3)), 
(1.6293,-0.9236,-0.6886)}. 

Vp oi8 = J? Poj8 =%U {(2^2/3, 0, -1/V3), 
(2^2/3, ±1/2,^3/6), (V6, 0,0)}. 

JSfp e>8 = J^ 8 =%U {(2 y/2/3, ±1/2, a/3/6), 
(2^/2/3, 0, -1/V3), (2^2/3, 1, -1/V3)}. 

%,i6 =%U {(1.3608, 5/6, 5/(6V5)), 

(1.3608, 0, -0.9623), (1.4628, -0.7513, 0.7110), 
(1.6024, -1.1134, -0.2106), (1.6293, 0.1346, 1.1442), 
(1.6293, 0.9236, -0.6887), (2^2j?>, 0, 0), 
(1.9336, -0.8075, -1.1032), (2.0380, 1.4396, 0.0642), 
(2.3097, 0.5202, 0.5210), (2.3097, 0.1911, -0.7110), 
(2.3499,-0.6462,0.2616)}. 



^p ,i6 = ^p ,8 u {(1-6293, -0.1345, 1.1442), 
(1.6293, 1.0582, -0.4556), (2y/2/3, 0, 0), 
(2.0380, 0.6643, -1.2789), (2.0380, -1.4396, 0.0642), 
(2.0380, 0.7754, 1.2147), (2.1187, 1.4645, 0.3160), 
(2.1187,-1.0059,1.1103)}. 

^Po ,ie = *4 U {(1.6279, 0.8995, -0.7184), 
(1.6279,-0.4977,1.0379), 

(1.6270, -0.9003, -0.7162), (1.6300, 0.5022, 1.0374), 
(1.6310,-0.0010,-1.1533), 

(1.6313, -1.1242, 0.2584), (1.6328, 1.1259, 0.2557), 
(2 y/2/3, 0, 0), (2.4495, 0, 1/V5), 
(2.4495, ±1/2, - V5/6), (3.2660, 0, 0)}. 

2^2/3,0,(2/3)^), 

(2^2/3,-1, -V3/3), (V6, 0,0), 

(>/6, -1/2, V3/2), (V6, ±1/2, -V3/2), 

(V6,±l,0)}. 
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